Place-based information processing method and apparatus

ABSTRACT

Methods and apparatuses for place-based information processing are disclosed. The method includes receiving annotation information inputted by a user; acquiring geographical location information of a mobile terminal device; sending the annotation information and the geographical location information to a server so that the server determines place information of where the user is located according to the geographical location information, establishes and stores a correspondence between the annotation information and the place information to provide the annotation information to a querying user in the place. In the present disclosure, “traces” of the user may be kept in the place and are not subject to an actual condition in the place, which do not need real paper and pen, and do not affect the actual scenery of the place.

CROSS REFERENCE TO RELATED PATENT APPLICATIONS

This application claims foreign priority to Chinese Patent ApplicationNo. 201410253628.2 filed on Jun. 9, 2014, entitled “Place-BasedInformation Processing Method and Apparatus,” and is a continuation ofU.S. patent application Ser. No. 14/733,294 filed Jun. 8, 2015, entitled“Place-Based Information Processing Method and Apparatus,” the contentsof both of which are hereby incorporated by reference in their entirety.

TECHNICAL FIELD

The present disclosure relates to the technical field of applications inmobile terminal devices, and particularly to methods and apparatuses ofplace-based information processing.

BACKGROUND

In real life, people often have the desire to leave a message at acertain place. For example, when dining in a restaurant, people usuallycomment on some dishes, express their mood during meals, or recordmeaningful events of the present day, etc. Therefore, some restaurantowners provide some stickers and pens for customers, so that thecustomers can write their comments, mood, events or the like on a pieceof paper, and put them on the wall of the restaurant. Thereafter, acustomer who dines in the restaurant can view messages left by othercustomers, or the same customer can view messages left previously byhimself/herself when eating in the same restaurant. There are many othersuch places. For example, when visiting a certain tourist attraction,sometimes a tourist may desire to leave a message or leave his/her nameto prove that he/she has been there, for example, “someone was here”,etc.

However, in a situation where a restaurant owner does not provide anypaper and pen or does not allow customers to put thereof on a wall, thecustomers cannot express messages that they want to write down in therestaurant timely. For a tourist attraction, no tourist will normallycarve his/her name, etc., at a scenic spot nowadays for the purpose ofscenery protection. This, however, results in a failure of meeting thetourist's desire to make records such as “having been there”.

Therefore, when a user needs to leave a message at a certain place, howto help the user so that the user may leave a message at any time andany place without being subject to constraints of the place of writingbecomes a technical problem to be solved by one skilled in the art.

SUMMARY

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify all key featuresor essential features of the claimed subject matter, nor is it intendedto be used alone as an aid in determining the scope of the claimedsubject matter. The term “techniques,” for instance, may refer todevice(s), system(s), method(s) and/or computer-readable instructions aspermitted by the context above and throughout the present disclosure.

The present disclosure provides a method and an apparatus of place-basedinformation processing, so that “traces” of a user may be kept in aplace without being subject to a real condition in the specific place,which does not need real paper and pen and does not affect actualscenery of the place.

The present disclosure provides the following solutions.

A place-based information processing method includes: receivingannotation information inputted by a user; acquiring information of ageographical location where a mobile terminal device is located; andsending the annotation information and the information of thegeographical location to a server to enable the server to determineinformation of a place where the user is located based on theinformation of the geographical location, establish and store acorrespondence relationship between the annotation information and theinformation of the place for providing the annotation information to aquerying user in the place.

A place-based information processing method includes: receiving arequest from a querying user to view surrounding annotation information;acquiring geographical location information of the querying user;sending the geographical location information to a server to enable theserver to determine a target place where the querying user is locatedbased on the geographical location information, determine and returneach piece of annotation information corresponding to the target placebased on a pre-stored correspondence relationship between each piece ofthe annotation information and place information; and providing eachpiece of the annotation information returned by the server to thequerying user.

A place-based information processing method includes: receivingannotation information uploaded by an annotating client and geographicallocation information of a user mobile terminal device; determininginformation of a place where the user is located based on thegeographical location information; and creating and storing acorrespondence relationship between the annotation information and theinformation of the place to enable determination of a target place wherea querying user based on the geographical location information includedin a request of querying for annotation information and returningannotation information corresponding to the target place to the queryingclient in response to receiving the request from the querying client.

A place-based information processing method includes: receiving arequest sent by a querying client to query for annotation information,the request including geographical location information of a queryinguser; determining a target place where the querying user is locatedaccording to the geographical location information; and determiningannotation information corresponding to the target place based on apre-created correspondence relationship between the annotationinformation and place information, and returning thereof to the queryingclient.

A place-based information processing apparatus includes: an annotationinformation receiving unit to receive annotation information inputted bya user; a first geographical location information acquisition unit toacquire geographical location information of a mobile terminal device;and a first sending unit to send the annotation information and thegeographical location information to a server to enable the server todetermine place information where the user is located based on thegeographical location information, establish and store a correspondencerelationship between the annotation information and the placeinformation for providing the annotation information to a querying userin the place.

A place-based information processing apparatus includes: a view requestreceiving unit to receive a request sent by a querying user to viewsurrounding annotation information; a second geographical locationinformation acquisition unit to acquire geographical locationinformation of the querying user; a second sending unit to send thegeographical location information to a server to enable the server todetermine a target place where the querying user is located according tothe geographical location information, to determine and return eachpiece of the annotation information corresponding to the target placeaccording to a pre-stored correspondence relationship between each pieceof the annotation information and information of the place; and anannotation information provision unit, configured to provide each pieceof the annotation information returned by the server to the queryinguser.

A place-based information processing apparatus includes: an informationreceiving unit to receive annotation information uploaded by anannotating client and geographical location information of a user mobileterminal device; a place information determination unit to determineplace information where the user is located according to thegeographical location information; and a storage unit to establish andstore a correspondence between the annotation information and the placeinformation to enable determination of a target place where a queryinguser is located based on geographical location information included in arequest of querying for annotation information and a return ofannotation information corresponding to the target place to the queryingclient in response to receiving the request from the querying client.

A place-based information processing apparatus includes: a requestreceiving unit to receive a request sent by a querying client to queryfor annotation information, the request carrying geographical locationinformation of a querying user; a target place determination unit todetermine a target place where the querying user is located according tothe geographical location information; and an annotation informationreturning unit to determine annotation information corresponding to thetarget place according to a pre-established correspondence betweenannotation information and place information, and to return thereof tothe querying client.

According to embodiments provided by the present disclosure, the presentdisclosure has the following technical effects:

Using the embodiments of the present disclosure, a user may utilize anapplication installed in a mobile terminal device to achieve annotationof specific information content in a place where the user is located,and such information content is not limited to text, but may alsoinclude a picture or audio, etc. “Traces” of the user may be kept in theplace via various forms of information content, and are not subject toactual conditions in the specific place, which does not need real paperand pen and will not affect the actual scenery of the place.

Apparently, any product implementing the present disclosure does notnecessarily achieve all of the above-mentioned advantages at the sametime.

BRIEF DESCRIPTION OF THE DRAWINGS

To describe the technical solutions in the embodiments of the presentdisclosure or existing technologies more clearly, example drawingsexemplifying embodiments are briefly introduced herein. Apparently, thedrawings in the following description merely represent some embodimentsof the present disclosure, and one of ordinary skill in the art mayfurther acquire other drawings according to those drawings withoutcreative efforts.

FIG. 1 is a flowchart of a first example method in accordance with thepresent disclosure.

FIG. 2 is a schematic diagram of a first user interface according to anembodiment of the present disclosure.

FIG. 3 is a schematic diagram of a second user interface according to anembodiment of the present disclosure.

FIG. 4 is a schematic diagram of a third user interface according to anembodiment of the present disclosure.

FIG. 5 is a flowchart of a second example method in accordance with thepresent disclosure.

FIG. 6 is a schematic diagram of a fourth user interface according to anembodiment of the present disclosure.

FIG. 7 is a schematic diagram of a fifth user interface according to anembodiment of the present disclosure.

FIG. 8 is a flowchart of a third example method in accordance with thepresent disclosure.

FIG. 9 is a flowchart of a fourth example method in accordance with thepresent disclosure.

FIG. 10 is a schematic diagram of a first example apparatus inaccordance with the present disclosure.

FIG. 11 is a schematic diagram of a second example apparatus inaccordance with the present disclosure.

FIG. 12 is a schematic diagram of a third example apparatus inaccordance with the present disclosure.

FIG. 13 is a schematic diagram of a fourth example apparatus inaccordance with the present disclosure.

DETAILED DESCRIPTION

The technical solutions in the embodiments of the present disclosurewill be clearly and fully described hereinafter with reference to theaccompanying drawings in the embodiments of the present disclosure.Apparently, the described embodiments merely represent some and not allof the embodiments of the present disclosure. All other embodimentsacquired by one of ordinary skill in the art based on the embodiments ofthe present disclosure belong to the protection scope of the presentdisclosure.

In the embodiments of the present disclosure, an application in a mobileterminal device (for example, a mobile terminal such as a mobile phone,a tablet computer, etc.) may be developed to meet the desire of a userto “put” information such as the mood he/she wants to express, the storyhe/she wants to share, etc., in a certain place. Thus, even if no paperor pen is provided at the place or pasting and carving are not allowed,the user may leave a “trace” at the place to indicate that he/she hasvisited there. Although being virtualized and not viewable directly witha naked eye, this type of “trace” may be viewed via a mobile terminaldevice of the user, which is equivalent to a physical existence and ismore in line with the trend of the information age. Exemplaryimplementations are described in detail hereinafter.

First Embodiment

Referring to FIG. 1, the first embodiment of the present disclosurefirst provides a place-based information processing method, which mayinclude the following method blocks:

S101 receives annotation information inputted by a user.

In an embodiment of the present disclosure, an executing entity of eachmethod block may be an application developed for a mobile terminaldevice, and the application may have a client-server architecture. Theclient may be installed in a mobile terminal device of a user, and isused for interacting with the user. The server runs in a backend server,and provides services such as a database aspect. In an implementation,the application may also be integrated in other application(s). Forexample, related functions in the embodiments of the present disclosuremay also be annotated in an existing instant messaging or socialnetworking application, etc. The first embodiment mainly introduces thesolution of the embodiments of the present disclosure from theperspective of a client.

For a client side of an application, an operation entry for the user toannotate information content may be provided no matter in what specificform the application may exist. For example, referring to FIG. 2, afterthe user enters the application, a user interface as shown in FIG. 2 maybe displayed to the user. If the user clicks on “Leave Your Information”in the user interface, a user interface as shown in FIG. 3 may beentered. An edit interface for editing information content may beprovided to the user in the user interface as shown in FIG. 3.Specifically, multiple input modes such as text input and voice inputmay be provided to the user. In other words, the annotation informationdescribed in the embodiment of the present disclosure refers to theinformation that the user needs to “leave” in the place where the useris located. This type of information may exist in various differentforms such as text and voice, and may establish an associationrelationship with the place where the user is located through theapplication in the terminal device. For example, controls such as akeyboard and an edit box for inputting text content may be provided in atext input/edit interface, and controls (for example, buttons) foradding elements such as pictures and emoticons may further be provided.After the user completes the input, for example, enters a text “NewArrival! Try this delicious coffee from Vietnam”, the user may click ona button “Where to put” in the user interface. Correspondingly, theclient side of the application may determine the information contentcurrently inputted by the user as the annotation information that needsto be left in the current place.

S102 obtains information of a geographical location of a mobile terminaldevice.

In response to determining the information content that the user desiresto annotate in the current place, information of a current geographicallocation of the user may further be determined. Apparently, in a realapplication, this method block S102 may also be performed prior to themethod block S101. For example, the geographical location information ofthe user may be acquired before the user enters specific annotationinformation, as long as the user accesses the client side of theapplication.

Specifically, many different approaches may exist for obtaining thegeographical location information. For example, existing mobile terminaldevices are generally provided with a positioning system such as GPS.Coordinates of the location of the user may be acquired via thepositioning system. In addition, positioning methods such ascommunications base station positioning and WIFI positioning may also beused to determine the geographical location information. Apparently, inan implementation, an operation entry may also be provided to the userfor manually entering the geographical location information thereof. Incases when the user does not enable the GPS positioning function or themobile terminal device thereof is not provided with the GPS positioningfunction, the user may edit the geographical location informationthereof manually.

S103 sends the annotation information and the geographical locationinformation to a server to enable the server to determine information ofa place where the user is located based on the geographical locationinformation, and to establish and store a correspondence relationshipbetween the annotation information and the place information forproviding the annotation information to a user who comes to the placeafterwards.

Upon receiving the annotation information and the geographical locationinformation of the user, the client side may upload these pieces ofinformation to the server, and the server may store these pieces ofinformation. Prior to storing, the server may further determine theplace where the user is located based on the geographical locationinformation first. As such, when storing is performed, a correspondencebetween the annotation information and the place is stored at the end,thereby implementing place-based information annotation and query.

The so-called “place” in the embodiments of the present disclosure maygenerally include some indoor places such as restaurants and coffeeshops, and may also include outdoor places such as tourist attractions.Normally, the locations of such places do not change and have anapparent regional boundary. Furthermore, the probabilities of changes indecorations, settings, objects, etc. in these places are relatively low.For these places, a map database may be created in advance to record acorrespondence between each place and a respective range of geographicalregion. As such, in response to receiving the current geographicallocation information of the user, a matching thereof against theregional range information of each place stored in the map database maybe performed. If the current location of the user is found to be withina range of a certain place, a determination may be made that the user iscurrently located in that place.

After determining information of the place where the user is currentlylocated, the server may store a correspondence between the annotationinformation and the place information in a server database. In this way,for the database of the server, multiple similar correspondencerelationships may be stored. If other users who comes to the place at alater time, the information content annotated in the place by this usermay be viewed. For example, such correspondence relationships may bestored in the database as shown in Table 1:

TABLE 1 Annotation information Place information Information content 1Place A Information content 2 Place B . . . . . .

A specific form of the information content may be a text, a picture, anaudio, etc., or may be a combination of multiple forms, for example, atext and a picture. Specifically, for example, when a user is dining ina restaurant, the user may express his/her mood in a form of a text, andat the same time take a photograph. In this case, the text forexpressing the mood and the photograph are equivalent to annotationinformation that the user wants to leave in the restaurant.

In an implementation, as the number of users who use such functionincreases, an amount of information content annotated by the users invarious places increases continuously, resulting in a continuousincrease in the scope of the database. In this case, the storedcorrespondence relationships may further be managed according tocorresponding places. For example, tables may be stored according toplaces, that is, information content annotated in a same place is storedin a same data table. Correspondingly, information content annotated indifferent places appears in different data tables, each placecorresponding to a data table. In this way, when subsequently providinga user in a certain place with information content annotated by otherusers, the place may be determined first, and a data table correspondingto the place is then found. Matched information content may be extractedand returned from the data table, thereby enabling a quick search.

In the above implementation, after a user annotates a certain piece ofannotation information in a certain place, content of the annotationinformation annotated by the user in the place (an implementation for aprocess of viewing annotation information will be described in detailhereinafter) may be viewed by another user using the client side of theapplication of the embodiment of the present disclosure that isinstalled in a mobile terminal device thereof when the other userarrives at that place.

As described above, an amount of information content annotated in a sameplace may be very large over time. In this case, how to provide aneffective information filtering/selection mechanism and thus how to makesuch function more interesting become problems that need to be furtherconsidered.

Accordingly, in an exemplary embodiment of the present disclosure, notonly the correspondence relationship between the annotation informationand the place information of the user may be recorded, but acorrespondence relationship between the annotation information andinformation of a specific annotation position in the place may also berecorded. For example, a certain user labels annotation information on acertain desk in a certain place. When viewing the annotation informationsubsequently, other users or the user will find that the annotationinformation is on the desk rather than appearing at other positions inthe place. This is closer to a real scenario of leaving a paper note orcarving words in a certain place. In an implementation, a correspondencerelationship between annotation information and a specific annotationposition may be very accurate, for example, having an accuracy up to aparticular point, or may be approximately accurate to a particularrange. Specific implementations are described in detail hereinafter.

First Implementation

In the first implementation, an annotation position of annotationinformation may be accurate to a particular area range. When positioninformation is recorded, the position information of annotation may bedescribed using image feature information in a scene area where anassociated placement position is located. Specifically, afterinformation content that is inputted by a user and needs to be annotatedis received, scene image information in the place may be collected.After the user triggers a put operation, currently collected imagefeature information may further be uploaded to the server whenannotation information and current geographical location information isuploaded. In this way, when recording a correspondence relationshipbetween the annotation information and the place, the server may recordthe corresponding image feature information into the correspondencerelationship.

For example, in order to collect scene image information in a place, animage capturing apparatus of a mobile terminal device of a user may beactivated automatically. In this way, an image of a scene in the placethat is captured by the image capturing apparatus appears in a userinterface of the mobile terminal device, and content of the image in theuser interface will continuously change as a viewing angle changes. Forexample, after the user clicks on the button “Where to put” as shown inFIG. 3, the image capturing apparatus may be activated. A display statusat a certain time point may be given as shown in FIG. 4. In a process ofdisplaying the captured image in the user interface, an operable objectmay further be provided to the user. For example, for a button “here” asshown in FIG. 4, an absolute position of the button in a screen mayremain unchanged. As the user changes the viewing angle of the imagecapturing apparatus, a relative position between the button and theimage may remain unchanged. After completing an adjustment of theviewing angle, the user may press the button. A client side of anapplication is notified via an operation of this operable object toannotate information content inputted thereby to an associated position.Correspondingly, for the client side of the application, after receivinga confirmation message for the annotation from the user, featureinformation of the image that is currently captured by the imagecapturing apparatus is extracted as image feature information of theannotation position. The image feature information of the annotationposition may then be sent to the server.

Specifically, when extracting the image feature information, featureinformation in multiple dimensions may be extracted. Examples include acolor of each pixel point in the image, a shape or contour of anidentifiable object in the image, a texture and a local feature of theimage, etc. The so-called identifiable object in the image may generallybe an object having a relative regular shape in the image, for example,an electrical appliance (such as a display device, a computer, atelevision, etc.), furniture (such as a desk, a chair, a stool, amessage board, for example), a work of art (such as a vase, adecoration, etc.), and may also be a structural construction such as astatue, a door, a wall, etc. In existing image recognition technologies,some or all parts of a shape and/or a contour of this type of object maybe recognized from a captured image. Furthermore, this type of objectnormally has the following features: (1) a position thereof in the placegenerally does not change; (2) a relative prominent labeling effect inrecognizing specific position information is inherent; and (3) whenannotating information, a user may find a certain object according tohabits in the real world, and annotate information onto or beside theobject. For example, when viewing an image of a current place through acamera, a user finds a display of a computer in the place, and thereforemay annotate information content to the upper right corner of thedisplay. Similarly, the user may also annotate information content on anobject having a relative regular shape such as an electrical appliance,appliance or a work of art, or may also annotate onto a structuralconstruction.

In short, when extracting image feature information at an annotationposition for each piece of annotation information, each piece of theannotation information may correspond to an image feature set, that is,an image is depicted using features in multiple dimensions. After theimage feature set is uploaded to the server, the server may add andstore thereof into a correspondence relationship between the annotationinformation and the place information as shown in Table 2.

TABLE 2 Image features of Annotation information Place informationannotation position Information content 1 Place A Image feature set 1Information content 2 Place A Image feature set 2 Information content 3Place B Image feature set 3 Information content 4 Place B Image featureset 4 Information content 5 Place B Image feature set 5 . . . . . . . ..

In this way, when providing annotation information to a querying user ata later time, the server may not only consider place information, butalso image feature information of a specific placement position of theannotation information. For example, when the information contentannotated in the place is provided to another user who visits the placeat a later time, an image capturing apparatus of that user may bestarted first to match feature information of an captured image againstimage features of the annotation position of each piece of theinformation content in the place. In response to a successful matching,corresponding information content is provided to that user. In otherwords, a user may see different information content in different captureranges in a same place. By filtering or selecting information contentaccording to a specific annotation position and a capture range, theuser may avoid receiving too much information, and the entireimplementation process becomes more interesting and is closer to a realscenario in the real world where “a ‘paper note’ annotated at a certainposition can only be seen at that position”.

Specifically, in order to implement a subsequent query operation, theserver may create indexes according to image features in variousdimensions after receiving an image feature set of each piece ofannotation information. Specifically, after receiving a query request,features in image information of a querying party may be extracted fromthe request, and an image feature set of the image information ismatched against the image feature set corresponding to each piece of theannotation information recorded in the database. When performing thematching, the image features may be matched in various dimensionsindividually. In order to improve a search efficiency, information ofthe features may be divided into primary feature information andsecondary feature information when extracting the information of thefeatures of the image. Generally, the primary feature information issimple and obvious information, for example, information such as acolor, a shape or a contour of an object in the image. The secondaryfeature information generally is in-depth information, includingtexture, local features, etc. As such, when providing informationcontent to other users at a later time, a primary index of theinformation content may first be created from each piece of theinformation content in a current place based on the primary featureinformation. After a preliminary search, a secondary index is createdbased on the secondary feature information. Matched information contentis found through searching at the end.

The first implementation may make annotation information accurate up toa range of a scene where a placement position thereof is located. Forexample, an image feature set of a certain piece of annotationinformation corresponds to an image A. When a querying user uploadsinformation of image features thereof, the image features correspond toan image B. The image A and the image B may be not identical, but adegree of similarity in terms of image features is relatively high. Assuch, the annotation information is provided to the querying user, andso on. Thus, a querying user may receive a number of pieces ofannotation information in a certain field of vision (because a conditionis considered to be met as long as a degree of similarity between imagefeature sets in the field of vision meets a certain requirement).However, since no further position information exists, when suchannotation information that meets the condition is displayed to thequerying user, each piece of the annotation information may be put atrandom in the image within the range of the field of vision of thequerying user. In other words, the annotation information may only bedisplayed in an approximate range.

Second Implementation

The foregoing first implementation may achieve an accuracy of a positionof placing annotation information up to a range. In the secondimplementation, in order to further improve the accuracy of a placementposition, further processing may be performed when image features arestored. Specifically, an identifiable object in an image may beextracted during image feature extraction, and a user usually selects anobject having relatively prominent features and places information at acertain position of the object when performing annotation. Therefore, inthe second implementation, if an image is found to include anidentifiable object on a basis of image feature extraction, anassociation relationship between annotation information and theidentifiable object may be recorded and sent to the server. In this way,when the correspondence relationship among the annotation information,the place and image feature set is stored, the server may add theassociation relationship with the identifiable object thereunto.

Specifically, the server may store a correspondence relationship asshown in Table 3 below:

TABLE 3 Image feature of Annotation information Place informationidentifiable object Information content 1 Place A Image feature ofidentifiable object 1 Information content 2 Place A Image feature ofidentifiable object 2 . . . . . . . . .

After the above correspondence relationships are stored according toTable 3, when annotation information is provided to a querying user, theannotation information may be displayed near an associated identifiableobject. Specifically, in response to receiving a request from a queryinguser, various types of image features in the first implementation may beused first to match features in an image captured by the querying useragainst image features of a respective identifiable object correspondingto each piece of annotation information, determine a matched piece ofannotation information, and return the matched piece of annotationinformation and image features of a corresponding identifiable object toa querying client. The querying client may locate the identifiableobject in the currently captured image, and provide each piece ofannotation information near the identifiable object. For example, a useris assumed to have annotated a piece of information near a certaindisplay device in a certain place. When an image capturing apparatus ofanother user captures the display device, that piece of annotationinformation may be provided to the viewing user and may still bedisplayed near the display device. Apparently, in this mode, a specificposition in the identifiable object where the annotation information islocated may not be ascertained. In other words, if a user putsannotation information at the upper right corner of a certain displaydevice, the annotation information may appear at the lower left cornerof the display device when the annotation information is finallyprovided to a querying user, which nevertheless is still displayed nearthe display device.

Third Implementation

In the third implementation, in order to achieve more accuratepositioning, when the annotation information is uploaded, a relativepositional relationship between the annotation information and theidentifiable object may be further determined, and information of therelative positional relationship may be uploaded to the server. In thisway, when storing the correspondence relationship among the annotationinformation, the place and the image features of the associatedidentifiable object, the server may add the information of the relativepositional relationship thereunto to achieve a matching with moreaccurate positioning when providing subsequently the annotationinformation to a querying user.

Specifically, when determining a relative positional relationshipbetween the annotation information and the identifiable object in theimage, the specific image may be determined first. When the userspecifies an action of placement confirmation, coordinates of aplacement point on a display screen of the terminal device may furtherbe recorded. Coordinates of each pixel in the captured image correspondto coordinates of the display screen. Therefore, knowing the coordinatesof the placement point on the display screen are known is equivalent toascertaining specific coordinates of the placement point in the capturedimage. An identifiable object may then be determined from the capturedimage. Since an object normally occupies multiple pixels, a pixel pointmay be selected from among the multiple pixel points to represent aposition of the identifiable object. For example, after defining an areawhere an identifiable object is located, the center of the area may beobtained using a certain algorithm, and the center is used forrepresenting the position of the identifiable object. A coordinatesystem may then be established using the center as an origin, andcoordinates of the annotation information in the coordinate system aredetermined based on the position of the annotation information in theimage. As such, the coordinates may be used to indicate the relativepositional relationship between the annotation information and theidentifiable object in the image, and may be sent to the server forstorage, as shown in Table 4:

TABLE 4 Annotation Place Image feature of Relative positionalinformation information identifiable object relationship InformationPlace A Object feature 1 Coordinates 1 content 1 Information Place AObject feature 2 Coordinates 2 content 2 . . . . . . . . . . . .

Coordinates in Table 2 above refer to coordinates of a respectiveposition point in which a piece of annotation information is locatedunder the coordinate system using the position point of an identifiableobject as the origin. It should be noted that the position of theidentifiable object may also be determined according to other rules. Forexample, not only the center of the area may be selected as describedabove, but the upper left corner, the lower right corner, etc., of thearea may also be selected. However, a specific selection mode is at bestto be consistent with a selection mode used during a subsequent query,thereby ensuring the accuracy of matching.

Using the second implementation, when another user comes to the placesubsequently and needs to view information content annotated in thesurrounding, a matching may be performed first using various types ofimage features in the first implementation if an image is captured viahis/her image capturing apparatus. Upon successful matching, anidentifiable object associated with each piece of matched annotationinformation in the image and corresponding coordinate information may bereturned to a querying client. The querying client determines a positionpoint of the identifiable object in a region where the identifiableobject is located in the captured image, and then determines a positionat which the annotation information needs to be displayed on theidentifiable object based on the coordinate information. In this way, ifa user annotates a piece of information near a certain display device ina certain place (for example, at the upper right corner), when an imagecapturing apparatus of another user captures the display device, theannotation information that is labeled at the upper right corner of thedisplay device may still be provided at the upper right corner of thedisplay device for the user to view if a high degree of similaritybetween other features in the image and the image features of theannotation position of this piece of information content is foundthrough comparison.

In the above implementation, the server adds the image featureinformation uploaded by the client side directly into the correspondencerelationship between the annotation information and the place. In thisway, when a user queries for annotation information at a later time,image information uploaded by the querying user may be matched againstthe image feature information corresponding to each piece of theannotation information. However, in the foregoing implementations, animage feature set corresponding to each piece of annotation informationneeds to be stored, which occupies a storage space of the server.Furthermore, as the size of the database increases continuously, thetime needed for a search also increases, which results in a slowresponse speed. In addition, in the foregoing implementations, theannotating user may select an annotation position at random. Althoughthe flexibility of the placement operation is improved, this may havethe following disadvantages. First, an error may occur during subsequentmatching if an image of a scene where an annotation position selected bythe user is located is not significantly distinctive. For example, aplace where the user annotates may be a white wall, and in this case,features extracted from the image are simple, and the annotationposition of the annotation information cannot be determined easilyaccording to the image features. Second, for the image captured by theuser and stored in the server side, the quality of the image may be poorand thus subsequent matching may be difficult due to factors such aspoor photographing skill of the user on one hand, and no accurate resultmay be acquired in the matching due to factors such as pixel, etc., interminal devices used by different users on the other hand. In view ofthe above factors, other implementations are provided in the embodimentsof the present disclosure, which are described below.

Fourth Implementation

In this implementation, the server may perform modeling for each placein advance, and establish an image feature library individually for eachplace. In the image feature library for each place, image features ofone or more landmark objects in the respective place may be stored. Thelandmark objects may be some furniture or electrical appliances, etc.,and these image features may be extracted from photographs of thelandmark objects that are captured in advance in the place. When takingphotographs of the landmark objects, objects around which user maydesire to put their annotation information may be predicted based on areal layout of the place, and these objects are then photographed asrepresentative landmark objects in the place, from which respectivefeatures are extracted. A respective set of the extracted image featuresis used to represent each landmark object. In short, the server maycreate an image feature library of landmark objects in advance as shownin Table 5:

TABLE 5 Place information Image feature of landmark object Place A Imagefeature of landmark object 1 Place A Image feature of landmark object 2. . . . . .

Each place may have multiple landmark objects, and image features ofeach landmark object may also be represented by a feature set inmultiple dimensions. In this way, in response to receiving annotationinformation uploaded by an annotating user and image feature informationof a corresponding scene, the server may first determine whether theimage of the scene includes content that is matched with imagefeature(s) of a landmark object, and if affirmative, may store acorrespondence relationship between the annotation information and anassociated place and add an identifier of the matched landmark objectinto the correspondence relationship. For example, if a certain placeincludes landmark objects such as a desk, a display device and an airconditioner, etc., and a certain user needs to put annotationinformation near the display device, a camera of a terminal device mayfocus on the display device. After a client side uploads the annotationinformation and image features of the place, the server analyzes theimage and detects that the image includes a landmark object—the displaydevice. As such, an identifier corresponding to the display device isread from the image feature library, and added into a currentcorrespondence relationship between the annotation information and theplace. An example is shown in Table 6.

TABLE 6 Place Annotation information information Identifier of landmarkobject Information content 1 Place A Landmark object 1 Informationcontent 2 Place A Landmark object 2 . . . . . . . . .

Subsequently when receiving a request from a querying user to viewannotation information in a certain place, a determination may be madeas to whether an image uploaded by the querying user includes a certainlandmark object in the current place. If included, annotationinformation corresponding to a matched landmark object may be extractedfrom an associated correspondence relationship and returned to thequerying user. When the annotation information is provided, theannotation information may also be displayed near the landmark object.As can be seen, the correspondence relationship recorded in Table 6above may achieve a similar or even better effect than thecorrespondence relationship recorded in Table 3 does, but a storagestructure of the former is simpler, which correspondence relationshipdoes not need to store image feature information and only needs to storethe associated landmark object.

In this mode, at which positions in a place the annotation informationmay be put are defined in advance, thereby avoiding problems that mayarise due to random annotations of users. In addition, an image featureset uploaded by a user for each piece of the annotation information doesnot need to be stored correspondingly, thus saving a storage space. Fora place, the number of landmark objects stored is not large and isfixed, so that the amount of work needed is basically fixed each timewhen an image feature matching is performed in response to a request forquerying the place. Apparently, in this mode, only an associationrelationship between annotation information and a landmark object may bedetermined, and a specific relative positional relationship therebetweenis not determined, however. Therefore, annotation information, whenbeing displayed to a querying user, may be displayed at random near alandmark object, thus potentially being inconsistent with an originalplacement position.

Fifth Implementation

In order to provide a more accurate relative positional relationshipbetween annotation information and a landmark object to a querying user,an approach similar to the one in the third implementation may also beused. In other words, coordinates of a placement point of a piece ofannotation information relative to an identifiable object are calculatedduring a process of uploading image feature information, and stored inthe server. Alternatively, in order to simplify this process, the fifthimplementation further provides an optimization method.

The fifth implementation is implemented on a basis of the fourthimplementation. Specifically, during the process of performing modelingfor each place, not only image features of landmark objects are storedfor each place, selectable annotation points may also be specified foreach landmark object and stored in an image feature library. A positionof each annotation point may also be represented by coordinates of therespective annotation point relative to such position as the center ofan associated landmark object. In this way, correspondence relationshipsstored in an image feature library may be given as shown in Table 7below:

TABLE 7 Place Image feature of Position of selectable informationlandmark object annotation point Place A Image feature of Coordinates ofannotation point landmark object 1 1 with respect to landmark object 1Place A Image feature of Coordinates of annotation point landmark object1 2 with respect to landmark object 1 . . . . . . . . .

Specifically, a same landmark object in a same place may includemultiple selectable annotation points, and coordinates of eachannotation point with respect to an associated landmark object areindividually recorded in an image feature library.

In this way, upon receiving an annotating request from an annotatinguser, a landmark object included in an image feature set uploaded by theannotating user is first recognized. Image features of the landmarkobject and relative coordinates of each selectable annotation point inthe landmark object are returned to an annotating client. As such, theannotating client may locate identifiable object(s) included in acurrently captured image based on the returned image features, displayicon(s) of selectable annotation point(s) at respective position(s)corresponding to the identifiable object(s) according to thecoordinates, and provide respective selection operation entr(ies). Inthis way, the annotating user may select an annotation point, and theclient side may upload an identifier of the annotation point selected bythe user to the server correspondingly, so that the server may addinformation about the annotation point selected by the user into acorrespondence relationship among the annotation information, anassociated place and the landmark object as shown in Table 8:

TABLE 8 Annotation Place Identifier of landmark Identifier ofinformation information object annotation point Information Place ALandmark object 1 Annotation point 1 content 1 Information Place ALandmark object 1 Annotation point 2 content 2 . . . . . . . . . . . .

Multiple pieces of annotation information may be labeled at differentpositions near a same landmark object of a same place. Records of otherlandmark objects and other places are omitted in Table 8 above.

In this approach, in response to receiving a subsequent request forquerying annotation information, the server may first determine whethera matched landmark object in scene image information uploaded by aquerying client exists, and if affirmative, may return each piece ofannotation information that is associated with the landmark object andcoordinate information of corresponding annotation points. Uponreceiving thereof, the querying client may determine a position at whicheach piece of annotation information is displayed based on the landmarkobject information and the coordinate information of the annotationpoints, and provide each piece of annotation information at thecorresponding position.

As can be seen, a final accuracy of matching between annotationinformation and an annotation position obtained in this approach issimilar to or even higher than that of the approach shown in Table 4.However, a storage structure as shown in Table 8 is simpler and mayimprove an efficiency of implementation.

A number of implementations are respectively described in detail above.In a real application, any of the above implementations may be selectedaccording to actual needs, which is not limited herein.

In addition, in an implementation, an operation of annotatinginformation content may generally be performed after a user has loggedin. In this case, identity information of the annotating user mayfurther be sent to the server, so that the server may add the identityinformation into a correspondence relationship for storage. In this way,the database stores the following information: what piece(s) ofinformation content is/are annotated in each place, which user(s)annotate(s) such piece(s) of information content, and at which specificposition(s) such piece(s) is/are annotated. Subsequently when theinformation content is provided to other users, not only the annotationposition information may be referenced, but a screening may also beperformed based on the identity information of the annotating user. Forexample, only a friend may view details of annotated informationcontent. Alternatively, only a user having the same preferences may viewthe information content, etc. Information about viewing may further befed back to the annotating user, for example, a count of the number offriends who have viewed the annotation information published by theuser, etc. Apparently, in a real application, a setting may include:users who are not friends are also allowed to view annotationinformation of each other. In this case, identity information of aviewer may further be displayed to the annotating user, and the user maybe prompted to add him/her as a friend, etc.

Under a circumstance that identity information of a user is stored,correspondence relationship in the database may be given as shown inTable 9:

TABLE 9 Place Image feature of Identifier of Information contentinformation annotation position annotating user Information Place AImage feature set 1 User X content 1 Information Place A Image featureset 2 User Y content 2 . . . . . . . . . . . .

In short, through the embodiment of the present disclosure, a user mayutilize an application installed in a mobile terminal device thereof toannotate specific information content in a place where the user islocated. This type of information content is not limited to text but mayalso be a picture or voice, etc. “Traces” of the user may therefore beleft in the place via various forms of information content. Such“traces” may reflect the feeling that has been expressed by the user orrecord what the user has seen and heard in the place in a certain amoment of the past, and are not subject to any real condition in thespecific place, which does not need real paper and pen and does notaffect the actual scenery of the place.

Second Embodiment

The foregoing first embodiment mainly describes the embodiments of thepresent disclosure from a perspective of how to annotate informationcontent in detail. The second embodiment mainly describes a perspectiveof how to query annotation information. Method blocks are stillperformed by a client side of an application. In a real application, theclient side may be the same as the client side mentioned in the firstembodiment. In other words, a same client side may implement a functionof annotating information in a place, and a function of viewingannotation information of other users in the place.

Referring to FIG. 5, the second embodiment of the present disclosureprovides a place-based information processing method, which may includethe following method blocks:

S501 receives a request from a querying user for viewing surroundingannotation information.

When a user starts a client side of an application, the client side mayreceive a request of the user to view annotation information in acurrent place. Apparently, after a user interface of the application isaccessed, respective operation entries of two functions may also beprovided in the interface. For example, referring to FIG. 2, besidesdisplaying an operation entry of “Leave Your Information”, an operationentry of “Find Surrounding” may also be displayed. Furthermore, anoperation of clicking on the operation entry of “Find Surrounding” bythe user in the user interface as shown in FIG. 2 may be taken as arequest of the user to view surrounding information content.

S502 obtains geographical location information of the querying user.

In response to receiving the request for viewing surrounding informationcontent, the geographical location information of the user may also beacquired first. Apparently, in a real application, the method block S502may also be performed before the method block S501. A specific processof implementation is similar to that in the first embodiment, and is notdescribed in detail herein.

S503 sends the geographical location information to a server to enablethe server to determine a target place where the querying user islocated based on the geographical location information, and to determineand return each piece of annotation information corresponding to thetarget place according to a pre-stored correspondence relationshipbetween the respective piece of annotation information and placeinformation.

Since a correspondence relationship between information contentannotated by each user and a place is stored in a database of the serverwhen users annotate information content, a request to view theinformation content annotated in the current place may be sent to theserver so that the server may acquire each piece of annotationinformation corresponding to the place and return thereof to thequerying client side upon determining information of the place.

S504 receives each piece of annotation information returned by theserver for the querying user to view.

Each piece of annotation information corresponding to the place, afterbeing acquired, may be provided to the current user. Specifically, suchinformation content may be provided in many different forms. Forexample, each piece of the information content may be presenteddirectly, or prompt information (for example, an icon, etc., withdifferent types of annotation information potentially using differenticons, for example, voice annotation information may be represented byan icon of a “horn”, and text annotation information may be representedby an icon of “the ellipses”, etc.) of each piece of the informationcontent may be provided in the user interface in the form of a listfirst. After the user clicks to view a certain piece of promptinformation, details of information content corresponding to that pieceof prompt information are presented to the user.

Apparently, if the database has further stored specific annotationposition information of the information content in the place, imageinformation in the place may also be captured first and then sent to theserver when providing the information content annotated in the place byother users to the user. When querying annotation information in theplace, the server may also find and return annotation information at acorresponding annotation position according to the image information.For example, if the server stores image feature informationcorresponding to each piece of annotation information, image featureinformation associated with an annotation position of each piece ofannotation information in the place where the user is currently locatedmay be acquired from the database. Furthermore, feature information of acurrently captured image is matched against the image featureinformation of the annotation position corresponding to each piece ofannotation information. Alternatively, image feature information of anidentifiable object that is associated with each piece of annotationinformation may be received from the server, and each piece ofannotation information is displayed near the identifiable object in thecurrently captured scene image accordingly. Alternatively, image featureinformation of an identifiable object that is associated with each pieceof annotation information and information of a relative positionalrelationship between each piece of annotation information and theidentifiable object may be received from the server. Each piece ofannotation information is then displayed at a position corresponding tothe identifiable object in the currently captured scene image.

When providing information content annotated by the other users in thecurrent place to a viewing user according to an image of a specificannotation position of the information content, another image layer maybe created on top of the captured image layer, and an identifier ofannotation information in a current field of vision is displayed at thisimage layer. For example, as shown in FIG. 6, after the user clicks onthe button “Find Surrounding Information” in FIG. 2, the image capturingapparatus of the mobile terminal device may be activated automatically.A background image in FIG. 6 is an image within a current range ofcapture, and identifier information 601, 602 and 603 displayed thereoncorresponds to identifier information of information content annotatedin the range of capture. If the user clicks on identifier informationcorresponding to the identifier information 601, details ofcorresponding information content may be presented. As shown in FIG. 7,the information content may include a user name of an annotating user,such as “Rachel”, and what the user annotates in the place is a piece ofvoice information. Accordingly, a playback button for playing the voiceinformation may further be provided in the details page.

If a search result corresponds to a failure of finding an image havingthe similar annotation position, which is equivalent to no other usershaving annotated information content in the current range of capture ofthe user, the user may be prompted to adjust a capturing angle andchange the range of capture in this case, as information contentannotated by users may be found in other ranges of capture.

In addition, in an implementation, for each user of an applicationprovided in the embodiments of the present disclosure, an accountthereof may normally be registered in the application, and a specificoperation of annotating information content or viewing informationcontent of other users may be performed after logging in using identityinformation for his/her own account. In other words, for each user,whether he/she is an annotator or a viewer of information content, anassociated system is capable of acquiring the identity information ofthe user. In addition, in a real application, a friendship relationshipmay be established among users. Therefore, in other embodiments of thepresent disclosure, information content may be viewed among friends.Specifically, information content annotated by a user in a certain placemay be viewed only by friends of the user. Correspondingly, when a userneeds to view information content annotated by other users in a certainplace, the user may only view information content annotated by his/herfriends in the place. In an implementation, when a correspondencerelationship between annotated information content and place informationis stored, identity information of an annotating user may also be storedas shown in Table 3. In this way, when a request of a certain user toview information content annotated in a current place is received,identity information of buddy users of the current user may be acquiredfirst, and identity information of an annotating user of each piece ofinformation content in the place where the user is currently located isfurther acquired from the database of the server. A determination isthen made as to whether a buddy user of the current viewing user isincluded therein. If affirmative, information content annotated by thebuddy user of the current user in the current place may be provided tothe current user.

Third Embodiment

The foregoing first embodiment and second embodiment describe thetechnical solutions of the present disclosure mainly from theperspective of a client side. A solution from the perspective of aserver is described hereinafter. In the third embodiment, details of animplementation process of a server side in a process of performingannotation by an annotating user are described.

Referring to FIG. 8, the third embodiment provides a place-basedinformation processing method, which may include the following methodblocks:

S801 receives annotation information uploaded by an annotating clientside and geographical location information of a user mobile terminaldevice.

S802 determines information of a place where the user is located basedon the geographical location information.

S803 creates and stores a correspondence relationship between theannotation information and the place information so that, when a requestsent by a querying client side to query the annotation information isreceived, a target place where the querying user is located isdetermined based on geographical location information included in therequest, and the annotation information corresponding to the targetplace is returned to the querying client side.

In order to acquire a precise annotation position to perform matchingmore accurately, the server may further receive annotation positioninformation of the annotation information in the place that is uploadedby the annotating client side, add and store the annotation positioninformation into the correspondence relationship. In this way, afterreceiving the request for querying the annotation information, theserver may match scene position information uploaded by the queryingclient side against the annotation position information corresponding toeach piece of annotation information in the target place, and return thematched annotation information to the querying client side.

In an implementation, the annotation position information may be imagefeature information of a scene image in the place that is uploaded bythe annotating client side. Alternatively, the annotation positioninformation may be image feature information of an identifiable objectincluded in a scene image in the place that is uploaded by theannotating client side, so that the position of the annotationinformation may be accurate to up a point near a specific identifiableobject. Alternatively, the annotation position information may be imagefeature information of an identifiable object included in a scene imagein the place uploaded by the annotating client side and relativepositional relationship information between the annotation informationand the identifiable object.

In another implementation, the server may create a first image featurelibrary for the place in advance. The first image feature library storesan identifier of at least one landmark object in the place andcorresponding image feature(s). In this case, when image featureinformation of a scene image in the place that is uploaded by anannotating client side is received, a determination may be made as towhether a certain landmark object in the place is included. Ifaffirmative, an identifier of that landmark object is taken as theannotation position information of the annotation information in theplace.

Alternatively, a second image feature library may be created for theplace in advance. The second image feature library stores an identifierof at least one landmark object in the place corresponding imagefeature(s), and relative coordinate information of at least oneselectable annotation point in each landmark object relative to therespective landmark object. In this case, when image feature informationof a scene image in the place that is uploaded by an annotating clientside is received, a determination may be made as to whether a certainlandmark object in the place is included therein. If affirmative, imagefeature(s) of that landmark object and relative coordinate informationof annotation point(s) are returned. After receiving information of aselected annotation point that is uploaded by the annotating clientside, the identifier of the landmark object and the information of theannotation point are taken as the annotation position information of theannotation information.

Fourth Embodiment

The fourth embodiment mainly describes an implementation of the serverside from the perspective of supporting enquiries of a querying party indetail.

Referring to FIG. 9, the fourth embodiment provides a place-basedinformation processing method, which may include the following methodblocks:

S901 receives a request for querying annotation information from aquerying client side, the request including geographical locationinformation of a querying user.

S902 determines a target place where the querying user is located basedon the geographical location information.

S903 determines annotation information corresponding to the target placeaccording to a pre-established correspondence relationship between theannotation information and place information, and returns thereof to thequerying client side.

The correspondence relationship may further include annotation positioninformation of the annotation information in the place. In this case,scene position information of the querying user in the current place maybe further acquired. Matched annotation information is then acquiredbased on annotation position information corresponding to each piece ofthe annotation information, and the matched annotation information isreturned to the querying client side to enable the querying client sideto provide the annotation information at the corresponding annotationposition.

Specifically, the annotation position information may be image featureinformation of a scene image in the place that is uploaded by anannotating client side. In this case, image feature information of ascene image in the place that is uploaded by the querying client sidemay be received, and the received image feature information is comparedwith image feature information corresponding to each piece of annotationinformation to acquire matched annotation information.

Alternatively, the annotation position information may be image featureinformation of an identifiable object in the place. In this case, imagefeature information of a scene image in the place that is uploaded bythe querying client side may be received. A determination may then bemade as to whether an identifiable object exists based on the receivedimage feature information. If affirmative, image feature(s) of theidentifiable object is/are compared with image feature(s) of arespective identifiable object that corresponds to each piece ofannotation information to acquire matched annotation information. Theimage feature information of the identifiable object may be uploaded bythe annotating client side, or may be stored in an image feature librarycreated in advance by the server side, and an associated correspondencerelationship stores an identifier of the identifiable object.

The annotation position information may also include image featureinformation of an identifiable object in the place, and relativepositional information between the annotation information and theidentifiable object. In this case, image feature information of a sceneimage in the place that is uploaded by the querying client side may bereceived, and a determination is made as to whether an identifiableobject exists based on the received image feature information. Ifaffirmative, image feature(s) of the identifiable object is/are comparedwith image feature(s) of a respective identifiable object thatcorresponds to each piece of annotation information to acquire matchedannotation information. The image feature information of theidentifiable object corresponding to the matched annotation informationand corresponding relative positional relationship information isreturned to the querying client side. The image feature information ofthe identifiable object and the relative positional relationshipinformation may be uploaded by the annotating client side, or may bestored in an image feature library established in advance by the serverside. An associated correspondence relationship stores an identifier ofthe identifiable object and an identifier of an annotation point.

It should be noted that implementation details of the foregoing second,third and fourth embodiments may be referenced to the description in thefirst embodiment, which are not described in detail herein.

Corresponding to the place-based information processing method providedin the first embodiment of the present disclosure, an embodiment of thepresent disclosure further provides a place-based information processingapparatus 1000. Referring to FIG. 10, the apparatus 1000 may include:

an annotation information receiving unit 1001 to receive annotationinformation inputted by a user;

a first geographical location information acquisition unit 1002 toacquire geographical location information of a mobile terminal device;and

a first sending unit 1003 to send the annotation information and thegeographical location information to a server to enable the server todetermine information of a place where the user is located based on thegeographical location information, to create and store a correspondencerelationship between the annotation information and the information ofthe place for providing the annotation information to a querying user inthe place.

In an implementation, in order to provide the annotation information ata more accurate position, the apparatus 1000 may further include:

an annotation position information sending unit 1004 to send informationof an annotation position of the annotation information in the place tothe server for adding the information of the annotation position intothe correspondence relationship for storage, and provide the annotationinformation to the querying user who is in the place and whose field ofvision includes the annotation position.

The annotation position information sending unit 1004 may include:

a scene image information capturing subunit 1005 to capture scene imageinformation in the place before or after receiving the annotationinformation inputted by the user; and

an image feature information sending subunit 1006 to extract featureinformation of the captured scene image in response to receiving anannotation confirmation message, and send the feature information to theserver as the annotation position information.

The server adds the received image feature information into thecorrespondence relationship for storage.

Alternatively, the server creates a first image feature library for theplace in advance. The first image feature library stores an identifierof at least one landmark object in the place and a corresponding imagefeature. After receiving an image feature of the scene image, the serverdetermines whether a certain landmark object in the place is includedtherein. If affirmative, the identifier of the landmark object is addedinto the correspondence relationship for storage.

Alternatively, the server establishes a second image feature library forthe place in advance. The second image feature library stores anidentifier of at least one landmark object in the place, a correspondingimage feature, and relative coordinate information of at least oneselectable annotation point in each landmark object relative to therespective landmark object. After receiving an image feature of thescene image, the server determines whether a certain landmark object inthe place exists therein. If affirmative, an image feature of thelandmark object and relative coordinate information of an associatedannotation point are returned.

The apparatus 1000 may further include:

a landmark object locating unit 1007 to locate the landmark object inthe currently captured scene image based on the received image featureof the landmark object, and display each selectable annotation point ontop of the scene image according to the relative coordinate information;and

an annotation point information sending unit 1008 to send informationabout a selected annotation point to the server after receiving aselection operation of the user to allow the server to add and store theidentifier of the landmark object and the information of the selectedannotation point into the correspondence relationship as the annotationposition information of the annotation information.

In another implementation, the annotation position information sendingunit 1004 may include:

a first image information capturing subunit 1009 to capture scene imageinformation in the place before or after the annotation informationinputted by the user is received;

a first feature information extraction subunit 1010 to extract featureinformation of the currently captured scene image in response toreceiving an annotation confirmation message;

a first determination subunit 1011 to determine whether the featureinformation includes an image feature of an identifiable object, andsend the image feature of the identifiable object to the server as theannotation position information if affirmative.

Alternatively, in another implementation, the annotation positioninformation sending unit 1004 may include:

a second image information capturing subunit 1012 to capture scene imageinformation in the place before or after the annotation informationinputted by the user is received;

a second feature information extraction subunit 1013 to extract featureinformation of the currently captured scene image in response toreceiving an annotation confirmation message; and

a second determination subunit 1014 to determine whether the featureinformation includes an image feature of an identifiable object;

a relative positional relationship acquisition subunit 1015 to acquirerelative positional relationship information between the annotationinformation and the identifiable object in response to a determinationresult of the second determination subunit being affirmative;

a sending subunit 1016 to send image feature information of theidentifiable object and the relative positional relationship informationto the server as the annotation position information.

In an implementation, the relative positional relationship acquisitionsubunit 1015 may determine a position of a position point that isrepresentative the identifiable object in the scene image, determine aposition of an annotation position point of the annotation informationon a display screen of the terminal device according to an operation ofthe user, use the position point of the identifiable object as an originto establish a coordinate system, determine coordinates of theannotation position point in the coordinate system, and use thecoordinates as the relative positional relationship information betweenthe annotation information and the identifiable object.

In a real application, the apparatus 1000 may further include:

a user identity information acquisition unit 1017 to acquire identityinformation of the user;

a user identity information sending unit 1018 to send the identityinformation of the user to the server to allow the server to add andstore the identity information into the correspondence relationship.

In an embodiment, the apparatus 1000 may further include one or moreprocessors 1019, an input/output interface 1020, a network interface1021 and memory 1022.

The memory 1022 may be a form of computer readable media, e.g., anon-permanent storage device, random-access memory (RAM) and/or anonvolatile internal storage, such as read-only memory (ROM) or flashRAM. The memory is an example of computer readable media. The computerreadable media may include a permanent or non-permanent type, aremovable or non-removable media, which may achieve storage ofinformation using any method or technology. The information may includea computer-readable command, a data structure, a program module or otherdata. Examples of computer storage media include, but not limited to,phase-change memory (PRAM), static random access memory (SRAM), dynamicrandom access memory (DRAM), other types of random-access memory (RAM),read-only memory (ROM), electronically erasable programmable read-onlymemory (EEPROM), quick flash memory or other internal storagetechnology, compact disk read-only memory (CD-ROM), digital versatiledisc (DVD) or other optical storage, magnetic cassette tape, magneticdisk storage or other magnetic storage devices, or any othernon-transmission media, which may be used to store information that maybe accessed by a computing device. As defined herein, the computerreadable media does not include transitory media, such as modulated datasignals and carrier waves.

In an embodiment, the memory 1022 may include program units 1023 andprogram data 1024. The program units 1023 may include one or more of theforegoing units and/or subunits as described in the foregoingembodiments of the apparatus 1000.

Corresponding to the place-based information processing method providedin the second embodiment of the present disclosure, an embodiment of thepresent disclosure further provides a place-based information processingapparatus 1100. Referring to FIG. 11, the apparatus 1100 may include:

a view request receiving unit 1101 to receive a request for viewingsurrounding annotation information from a querying user;

a second geographical location information acquisition unit 1102 toacquire geographical location information of the querying user;

a second sending unit 1103 to send the geographical location informationto a server to enable the server to determine a target place where thequerying user is located based on the geographical location information,and to determine and return each piece of the annotation informationcorresponding to the target place based on a pre-stored correspondencerelationship between the respective piece of the annotation informationand place information;

an annotation information provision unit 1104 to provide each piece ofthe annotation information returned by the server to the querying user.

The server further stores annotation position information correspondingto each piece of the annotation information. The apparatus 1100 mayfurther include:

an image information capturing and sending unit 1105 to capture and sendscene image information in the place to the server to facilitate theserver to determine and return annotation information that is in theplace and within a field of vision of the querying user according to thescene image information received thereby.

In an implementation, the apparatus 1100 may further include:

an identifiable object information receiving unit 1106 to receive imagefeature information of an identifiable object that is associated witheach piece of the annotation information returned by the server, anddisplay each piece of the annotation information near the identifiableobject in the currently captured scene image accordingly.

In addition, the apparatus 1100 may further include:

a relative positional relationship information receiving unit 1107 toreceive image feature information of an identifiable object that isassociated with each piece of the annotation information returned by theserver and relative positional relationship information between eachpiece of the annotation information and the identifiable object, anddisplay each piece of the annotation information at a positioncorresponding to the identifiable object in the currently captured sceneimage accordingly.

In addition, the apparatus 1100 may further include:

a first prompt information provision unit 1108 to provide first promptinformation if the information returned by the server does not includematched annotation information, the first prompt information being usedfor prompting the user to change the field of vision.

If the server further stores identity information of an annotating userof information content, the apparatus 1100 may further include:

an identity information sending unit 1109 to send identity informationof the querying user to the server to facilitate the server to acquireidentity information of buddy users of the querying user, to acquireidentity information of an annotating user of each piece of theinformation content in the place where the user is currently located,and to determine whether a friend user of the querying user exists; and

a returning unit 1110 to return annotation information of the buddy userof the querying user in the place if affirmative.

In addition, the apparatus 1100 may further include:

a second prompt information provision unit 1111 to provide second promptinformation to the querying user, the second prompt information beingused for indicating that the querying user is allowed to leaveannotation information in the current place.

In an embodiment, the apparatus 1100 may further include one or moreprocessors 1112, an input/output interface 1113, a network interface1114 and memory 1115. The memory 1115 may include a form ofcomputer-readable media as described in the foregoing description.

In an embodiment, the memory 1115 may include program units 1116 andprogram data 1117. The program units 1116 may include one or more of theforegoing units and/or subunits as described in the foregoingembodiments of the apparatus 1100.

Corresponding to the place-based information processing method providedin the third embodiment of the present disclosure, an embodiment of thepresent disclosure further provides a place-based information processingapparatus. Referring to FIG. 12, the apparatus 1200 may include:

an information receiving unit 1201 to receive annotation informationuploaded by an annotating client side and geographical locationinformation of a user mobile terminal device;

a place information determination unit 1202 to determine placeinformation about where the user is located according to thegeographical location information; and

a storage unit 1203 to establish and store a correspondence relationshipbetween the annotation information and the place information to allowdetermining a target place where a querying user is located based ongeographical location information included in a request for querying theannotation information, and returning annotation informationcorresponding to the target place to a query client side upon receivingthe request from the querying client side to query is received.

In an implementation, the apparatus 1200 may further include:

an annotation position information receiving unit 1204 to receiveannotation position information of the annotation information in theplace that is uploaded by the annotating client side, to add theannotation position information into the correspondence relationship forstorage, and to provide the annotation information to a querying userwho is in the place and whose field of vision includes the annotationposition.

The annotation position information includes image feature informationof a scene image in the place that is uploaded by the annotating clientside.

Alternatively, the annotation position information includes imagefeature information of an identifiable object included in a scene imagein the place that is uploaded by the annotating client side.

Still alternatively, the annotation position information includes imagefeature information of an identifiable object included in a scene imagein the place uploaded by the annotating client side, and relativepositional relationship information between the annotation informationand the identifiable object.

In an implementation, a first image feature library may be establishedfor the place in advance, the first image feature library storing anidentifier of at least one landmark object in the place and acorresponding image feature. The annotation position informationreceiving unit 1204 may include:

a first image feature information receiving subunit 1205 to receiveimage feature information of a scene image in the place that is uploadedby the annotating client side;

a first determination subunit 1206 to determine whether a landmarkobject in the place exists in the image feature information, and to setan identifier of the landmark object as the annotation positioninformation of the annotation information in the place if affirmative.

Alternatively, a second image feature library is established for theplace in advance, the second image feature library storing an identifierof at least one landmark object in the place, a corresponding imagefeature, and relative coordinate information of at least one selectableannotation point in each landmark object relative to the respectivelandmark object. The annotation position information receiving unit 1204may include:

a second image feature information receiving subunit 1207 to receiveimage feature information of a scene image in the place that is uploadedby the annotating client side;

a second determination subunit 1208 to determine whether a landmarkobject in the place exists in the image feature information, and returnan image feature of the landmark object and relative coordinateinformation of annotation point(s) thereof if affirmative;

an annotation point information receiving unit 1209 to receiveinformation of a selected annotation point that is uploaded by theannotating client side, and to set the identifier of the landmark objectand the information of the annotation point as the annotation positioninformation of the annotation information.

In an embodiment, the apparatus 1200 may further include one or moreprocessors 1210, an input/output interface 1211, a network interface1212 and memory 1213. The memory 1213 may include a form ofcomputer-readable media as described in the foregoing description.

In an embodiment, the memory 1213 may include program units 1214 andprogram data 1215. The program units 1214 may include one or more of theforegoing units and/or subunits as described in the foregoingembodiments of the apparatus 1200.

Corresponding to the place-based information processing method providedin the fourth embodiment of the present disclosure, an embodiment of thepresent disclosure further provides a place-based information processingapparatus. Referring to FIG. 13, the apparatus 1300 may include:

a request receiving unit 1301 to receive a request for queryingannotation information from a querying client side, the requestincluding geographical location information of a querying user;

a target place determination unit 1302 to determine a target place wherethe querying user is located according to the geographical locationinformation; and

an annotation information returning unit 1303 to determine, according toa pre-established correspondence relationship between the annotationinformation and place information, the annotation informationcorresponding to the target place, and return the annotation informationto the querying client side.

The correspondence relationship further includes annotation positioninformation of the annotation information in the place. The apparatus1300 may further include:

a scene position information acquisition unit 1304 to acquire sceneposition information of the querying user in the current place;

a matched annotation information acquisition unit 1305 to acquirematched annotation information according to respective annotationposition information corresponding to each piece of the annotationinformation, and to return the matched annotation information to thequerying client side to enable the querying client side to provide theannotation information at the respective annotation position.

The annotation position information includes image feature informationof a scene image in the place that is uploaded by an annotating clientside.

The scene position information acquisition unit 1304 may receive imagefeature information of a scene image in the place that is uploaded bythe querying client side.

The matched annotation information acquisition unit 1305 may compare thereceived image feature information with the image feature informationcorresponding to each piece of the annotation information, and acquirethe matched annotation information.

Alternatively, the annotation position information includes imagefeature information of an identifiable object in the place.

The scene position information acquisition unit 1304 may receive imagefeature information of a scene image in the place that is uploaded bythe querying client side.

The matched annotation information acquisition unit 1305 may determinewhether an identifiable object exists according to the received imagefeature information, to compare an image feature of the identifiableobject with an image feature of a respective identifiable objectcorresponding to each piece of the annotation information ifaffirmative, and to acquire the matched annotation information.

Still alternatively, the annotation position information includes imagefeature information of an identifiable object in the place and relativepositional relationship information between the annotation informationand the identifiable object.

The scene position information acquisition unit 1304 may receive imagefeature information of a scene image in the place that is uploaded bythe querying client side.

The matched annotation information acquisition unit 1305 may determinewhether an identifiable object exists according to the received imagefeature information, to compare an image feature of the identifiableobject with an image feature of a respective identifiable objectcorresponding to each piece of the annotation information, and toacquire the matched annotation information.

In this case, the apparatus 1300 may further return the image featureinformation of the identifiable object corresponding to the matchedannotation information and the relative positional relationshipinformation to the querying client side.

In an embodiment, the apparatus 1300 may further include one or moreprocessors 1306, an input/output interface 1307, a network interface1308 and memory 1309. The memory 1309 may include a form ofcomputer-readable media as described in the foregoing description.

In an embodiment, the memory 1309 may include program units 1310 andprogram data 1311. The program units 1310 may include one or more of theforegoing units and/or subunits as described in the foregoingembodiments of the apparatus 1300.

Through the embodiments of the present disclosure, a user may utilize anapplication installed in a mobile terminal device thereof to annotatespecific information content in a place where the user is located. Thistype of information content is not limited to text but may also be apicture or voice, etc. “Traces” of the user may therefore be left in theplace via various forms of information content. Such “traces” mayreflect the feeling that has been expressed by the user or record whatthe user has seen and heard in the place in a certain a moment of thepast, and are not subject to any real condition in the specific place,which does not need real paper and pen and does not affect the actualscenery of the place.

From the description of the foregoing embodiments, one skilled in theart can clearly understand that the present disclosure can beimplemented using software with a necessary general hardware platform.Based on this understanding, the essence of the technical solutions inthe present disclosure or the part contributing to the existingtechnologies may be implemented in the form of a software product. Thecomputer software product may be stored in a storage medium, such as aread-only memory (ROM)/a random access memory (RAM), a magnetic disk, oran optical disc, which includes multiple instructions for instructing acomputing device (which may be a personal computer, a server, a networkdevice, or the like) to perform the method described in the embodimentsor certain parts of the embodiments of the present disclosure.

The embodiments in the specification are described in a progressivemanner. Reference may be made to each other for the same or similarparts in the embodiments. An emphasis of each embodiment is differentfrom those of other embodiments. In particular, the apparatuses orapparatus embodiments are basically similar to the method embodimentsand are therefore described in a relative simple manner. For relevantpart, reference may be made to the description of the part of the methodembodiment. The systems or system embodiments described above are merelyexemplary. The units that are described as separate components may ormay not be physically separated. The components shown as units may ormay not be physical units, and may be located in a single location ormay be distributed among multiple network units. Some or all of themodules may be selected according to actual needs to achieve theobjectives of the solutions of the embodiments. One of ordinary skill inthe art can understand and implement the embodiments without making anycreative effort.

The methods and apparatuses of place-based information processing thatare provided by the present disclosure are described in detail above.Specific examples are used in the specification to explain theprinciples and implementations of the present disclosure. Thedescription of the foregoing embodiments, however, is merely used tofacilitate the understanding of the methods and core ideas of thepresent disclosure. One of ordinary skill in the art can makemodifications to the specific implementations and the application scopesbased on the ideas of the present disclosure. Therefore, the content ofthe specification should not be construed as a limitation to the presentdisclosure.

What is claimed is:
 1. A method comprising: receiving a request from aquerying user for viewing surrounding annotation information; obtaininggeographical location information of the querying user; sending thegeographical location information to a server to enable the server todetermine a target place where the querying user is located based on thegeographical location information, and to determine and return eachpiece of annotation information corresponding to the target placeaccording to a pre-stored correspondence relationship between therespective piece of annotation information and place information; andreceiving each piece of annotation information returned by the serverfor the querying user to view.
 2. The method of claim 1, wherein theserver further stores annotation position information corresponding toeach piece of the annotation information, and further comprising:capturing and sending scene image information in the place to the serverto enable the server to determine and return annotation information thatis in the place and within a field of vision according to the sceneimage information received thereby.
 3. The method of claim 2, furthercomprising: receiving from the server image feature information of anidentifiable object that is associated with each piece of annotationinformation, and displaying each piece of annotation information nearthe identifiable object in the currently captured scene imageaccordingly.
 4. The method of claim 2, further comprising: receivingfrom the server image feature information of an identifiable object thatis associated with each piece of annotation information and informationof a relative positional relationship between each piece of annotationinformation and the identifiable object, and displaying each piece ofannotation information at a position corresponding to the identifiableobject in the currently captured scene image.
 5. The method of claim 1,wherein the information returned by the server does not include matchedannotation information; and further comprising: providing first promptinformation to the querying user prompting the querying user to change afield of vision.
 6. The method of claim 1, wherein the server furtherstores identity information of an annotating user of informationcontent, and further comprising: sending identity information of thequerying user to the server to enable the server to acquire identityinformation of buddy users of the querying user, to acquire identityinformation of annotating users of each piece of the information contentin the place where the user is currently located, and to determine thata buddy user of the querying user is among the annotating users; andreturning annotation information of the buddy user of the querying userin the place.
 7. The method of claim 1, further comprising: providingsecond prompt information to the querying user indicating that thequerying user is allowed to leave annotation information in the currentplace.
 8. A method comprising: receiving a request for queryingannotation information from a querying client terminal, the requestincluding geographical location information of a querying user;determining a target place where the querying user is located based onthe geographical location information; and determining annotationinformation corresponding to the target place according to apre-established correspondence relationship between the annotationinformation and place information, and returning thereof to the queryingclient terminal.
 9. The method of claim 8, wherein the correspondencerelationship further includes annotation position information of theannotation information in the place, and further comprising: acquiringscene position information of the querying user in the current place;acquiring matched annotation information based on annotation positioninformation corresponding to each piece of the annotation information,and returning the matched annotation information to the querying clientterminal to enable the querying client terminal to provide theannotation information at the corresponding annotation position.
 10. Themethod of claim 9, wherein the annotation position information is imagefeature information of a scene image in the place that is uploaded by anannotating client terminal; wherein acquiring scene position informationof the querying user in the current place comprises: receiving imagefeature information of a scene image in the place that is uploaded bythe querying client terminal; and wherein acquiring matched annotationinformation based on annotation position information corresponding toeach piece of the annotation information comprises: comparing thereceived image feature information with image feature informationcorresponding to each piece of annotation information to acquire matchedannotation information.
 11. The method of claim 9, wherein theannotation position information is image feature information of anidentifiable object in the place; wherein acquiring scene positioninformation of the querying user in the current place comprises:receiving image feature information of a scene image in the place thatis uploaded by the querying client terminal; and wherein acquiringmatched annotation information based on annotation position informationcorresponding to each piece of the annotation information comprises:determining that an identifiable object exists based on the receivedimage feature information; and comparing at least one image feature ofthe identifiable object with at least one image feature of a respectiveidentifiable object that corresponds to each piece of annotationinformation to acquire matched annotation information.
 12. The method ofclaim 9, wherein the annotation position information includes imagefeature information of an identifiable object in the place and relativepositional information between the annotation information and theidentifiable object; wherein acquiring scene position information of thequerying user in the current place comprises: receiving image featureinformation of a scene image in the place that is uploaded by thequerying client terminal; and wherein acquiring matched annotationinformation based on annotation position information corresponding toeach piece of the annotation information comprises: determining that anidentifiable object exists based on the received image featureinformation; and comparing at least one image feature of theidentifiable object with at least one image feature of a respectiveidentifiable object that corresponds to each piece of annotationinformation to acquire matched annotation information; and furthercomprising: returning the image feature information of the identifiableobject corresponding to the matched annotation information andcorresponding relative positional relationship information to thequerying client terminal.
 13. The method of claim 12, wherein the imagefeature information of an identifiable object in the place and therelative position information are uploaded by an annotating clientterminal.
 14. An apparatus comprising: one or more processors; memory; aview request receiving unit stored in the memory and configured to beexecutable by the one or more processors to receive a request from aquerying user for viewing surrounding annotation information; ageographical location information acquiring unit stored in the memoryand configured to be executable by the one or more processors to obtaingeographical location information of the querying user; a sending unitstored in the memory and configured to be executable by the one or moreprocessors to send the geographical location information to a server toenable the server to determine a target place where the querying user islocated based on the geographical location information, and to determineand return each piece of annotation information corresponding to thetarget place according to a pre-stored correspondence relationshipbetween the respective piece of annotation information and placeinformation; and an annotation information providing unit stored in thememory and configured to be executable by the one or more processors toreceive each piece of annotation information returned by the server forthe querying user to view.
 15. The apparatus of claim 14, wherein theserver further stores annotation position information corresponding toeach piece of the annotation information, and further comprising: animage information capturing and sending unit stored in the memory andconfigured to be executable by the one or more processors to capture andsend scene image information in the place to the server to enable theserver to determine and return annotation information that is in theplace and within a field of vision according to the scene imageinformation received thereby.
 16. The apparatus of claim 15, furthercomprising: an identifiable object information receiving unit stored inthe memory and configured to be executable by the one or more processorsto receive from the server image feature information of an identifiableobject that is associated with each piece of annotation information, anddisplay each piece of annotation information near the identifiableobject in the currently captured scene image accordingly.
 17. Theapparatus of claim 15, further comprising: a relative positionalrelationship information receiving unit stored in the memory andconfigured to be executable by the one or more processors to receivefrom the server image feature information of an identifiable object thatis associated with each piece of annotation information and informationof a relative positional relationship between each piece of annotationinformation and the identifiable object, and display each piece ofannotation information at a position corresponding to the identifiableobject in the currently captured scene image.
 18. The apparatus of claim14, wherein the information returned by the server does not includematched annotation information; and further comprising: a first promptinformation providing unit stored in the memory and configured to beexecutable by the one or more processors to provide first promptinformation to the querying user prompting the querying user to change afield of vision.
 19. The apparatus of claim 14, wherein the serverfurther stores identity information of an annotating user of informationcontent, and further comprising: an identity information sending unitstored in the memory and configured to be executable by the one or moreprocessors to send identity information of the querying user to theserver to enable the server to acquire identity information of buddyusers of the querying user, to acquire identity information ofannotating users of each piece of the information content in the placewhere the user is currently located, and to determine that a buddy userof the querying user is among the annotating users; and a returning unitstored in the memory and configured to be executable by the one or moreprocessors to return annotation information of the buddy user of thequerying user in the place.
 20. The apparatus of claim 14, furthercomprising: a second prompt information providing unit stored in thememory and configured to be executable by the one or more processors toprovide second prompt information to the querying user indicating thatthe querying user is allowed to leave annotation information in thecurrent place.